Skip to content

Push down COALESCE in PostgreSQL connector#11535

Merged
ebyhr merged 1 commit intotrinodb:masterfrom
ebyhr:ebi/connector-expression-coalesce
Apr 6, 2026
Merged

Push down COALESCE in PostgreSQL connector#11535
ebyhr merged 1 commit intotrinodb:masterfrom
ebyhr:ebi/connector-expression-coalesce

Conversation

@ebyhr
Copy link
Copy Markdown
Member

@ebyhr ebyhr commented Mar 17, 2022

Description

Push down COALESCE in PostgreSQL connector

Release notes

# PostgreSQL
* Improve performance of queries involving `COALESCE` by pushing expression
  computation to the underlying database. ({issue}`11535`)

@findepi
Copy link
Copy Markdown
Member

findepi commented Mar 21, 2022

@ebyhr can you please rebase?

@ebyhr ebyhr force-pushed the ebi/connector-expression-coalesce branch from 4dff92a to 5bb372b Compare March 21, 2022 14:08
@ebyhr
Copy link
Copy Markdown
Member Author

ebyhr commented Mar 21, 2022

@findepi Rebased on upstream.

Copy link
Copy Markdown
Member

@findepi findepi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Translate COALESCE to connector expressions"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wendigo 's PR adds some shorthand function for this, am i right, @wendigo ?

no change requested now, but we should unify later

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes,I will note it down to unify that after those PRs are merged.

@ebyhr ebyhr force-pushed the ebi/connector-expression-coalesce branch from 5bb372b to 22099ed Compare March 21, 2022 23:17
@ebyhr ebyhr force-pushed the ebi/connector-expression-coalesce branch from 22099ed to c77604c Compare March 22, 2022 13:40
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TryExpression is desugared to try function and it's marked as non-deterministic.
We don't push non-deterministic predicates down into the connector, however, they are handled separately from non-translatable ones.

Thus, it's a wrong change, as it changes the semantics of the test case.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed to SearchedCaseExpression though I have a local patch to support it as connector expressions. Do you have other suggested expressions?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See #11511 (comment) and #11544 (comment)

I don't have a plan for testing things. @martint do you have one?
cc @wendigo @hashhar

@ebyhr ebyhr force-pushed the ebi/connector-expression-coalesce branch from c77604c to 6810da6 Compare March 23, 2022 01:25
@assaf2 assaf2 self-requested a review March 27, 2022 10:24
@ebyhr ebyhr force-pushed the ebi/connector-expression-coalesce branch from 6810da6 to 1cd9346 Compare March 28, 2022 00:08
@ebyhr ebyhr force-pushed the ebi/connector-expression-coalesce branch from 1cd9346 to d4722a4 Compare April 1, 2022 08:05
@ebyhr ebyhr requested a review from findepi April 5, 2022 09:13
@findepi
Copy link
Copy Markdown
Member

findepi commented Apr 5, 2022

@martint by any chance, does coalesce have lazy eval semantics?

cc @kasiafi

@martint
Copy link
Copy Markdown
Member

martint commented Apr 5, 2022

@martint by any chance, does coalesce have lazy eval semantics?

Yes, coalesce is described by the SQL spec as syntactic sugar for a CASE expression.

<case expression> ::=
    <case abbreviation>
  | <case specification>

<case abbreviation> ::=
    NULLIF <left paren> <value expression> <comma> <value expression> <right paren>
  | COALESCE <left paren> <value expression>
      { <comma> <value expression> }... <right paren>
d) COALESCE (V1, V2) is equivalent to the following <case specification>:

  CASE 
    WHEN V1 IS NOT NULL THEN 
      V1 
    ELSE
      V2 
  END

e) COALESCE (V1, V2, ..., Vn), for n ≥ 3, is equivalent to the following <case specification>:

  CASE 
    WHEN V1 IS NOT NULL THEN 
      V1 
    ELSE 
      COALESCE (V2, ..., Vn) 
  END

@findepi
Copy link
Copy Markdown
Member

findepi commented Apr 6, 2022

If i remember correctly, the CASE's conditions are semantically evaluated eagerly, but the values are conditional.
Since the COALESCE is defined as equivalent to recursive CASE (not flattened), it means the components are semantically evaluated lazily,

E.g. this should not fail

coalesce(CASE WHEN x = 0 THEN 1 ELSE NULL END, 10 / x)

similar to #11699 (comment), this means this should not be modeled as a function call.

OTOH, modeling COALESCE as a recursive CASE in ConnectorExpressions will be harder to consume within connectors.

@martint what about we model COALESCE as a new Coalesce class?

@ebyhr ebyhr force-pushed the ebi/connector-expression-coalesce branch from d4722a4 to 0d80cff Compare April 14, 2022 09:33
@ebyhr
Copy link
Copy Markdown
Member Author

ebyhr commented Apr 14, 2022

(Rebased on upstream to resolve conflicts)

@findepi
Copy link
Copy Markdown
Member

findepi commented Apr 14, 2022

OTOH, modeling COALESCE as a recursive CASE in ConnectorExpressions will be harder to consume within connectors.

@martint what about we model COALESCE as a new Coalesce class?

@martint and I talked about this and because coalesce features lazy eval, the current consensus is that should be modeled as a function that takes lazy value suppliers, i.e. lambdas. The alternative of implementing this as a dedicated form of a ConnectorExpression would work, but we would deviate from "have connector expressions as simple as possible" principle and was currently rejected.

However, lambdas support is not a trivial thing, and we don't think we're currently ready to work on this yet.

@ebyhr
Copy link
Copy Markdown
Member Author

ebyhr commented Apr 15, 2022

Thanks for your explanation. Let me close this PR.

@ebyhr ebyhr closed this Apr 15, 2022
@ebyhr ebyhr deleted the ebi/connector-expression-coalesce branch April 15, 2022 06:44
@raunaqmorarka
Copy link
Copy Markdown
Member

@ebyhr you can redo this now that #28984 has landed

@ebyhr ebyhr restored the ebi/connector-expression-coalesce branch April 3, 2026 23:58
@ebyhr ebyhr reopened this Apr 3, 2026
@github-actions github-actions bot added the postgresql PostgreSQL connector label Apr 3, 2026
@ebyhr ebyhr force-pushed the ebi/connector-expression-coalesce branch from 0d80cff to 72651ec Compare April 6, 2026 05:35
@raunaqmorarka
Copy link
Copy Markdown
Member

@CodeRabbit review

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Apr 6, 2026

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Apr 6, 2026

📝 Walkthrough

Walkthrough

A new RewriteCoalesce connector expression rewrite rule was introduced that handles COALESCE function calls by validating they contain at least two arguments, rewriting each argument, and constructing a ParameterizedExpression with SQL text COALESCE(%s). The PostgreSQL plugin was updated to register this rule in its expression rewriter configuration. Test coverage was extended to verify the rule correctly translates COALESCE expressions with both column references and constants, and to validate that COALESCE predicates are fully pushed down to the database during query execution.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (2)
plugin/trino-base-jdbc/src/main/java/io/trino/plugin/jdbc/expression/RewriteCoalesce.java (2)

38-46: Simplify pattern: the capture is unnecessary.

The CALL capture is not needed since the call parameter passed to rewrite() is the same matched Call object. You can remove the capture and use call directly in the loop.

♻️ Suggested simplification
 public class RewriteCoalesce
         implements ConnectorExpressionRule<Call, ParameterizedExpression>
 {
-    private static final Capture<Call> CALL = newCapture();
     private final Pattern<Call> pattern;
 
     public RewriteCoalesce()
     {
         this.pattern = call()
-                .with(functionName().matching(name -> name.equals(COALESCE_FUNCTION_NAME)))
-                .capturedAs(CALL);
+                .with(functionName().equalTo(COALESCE_FUNCTION_NAME));
     }

Also update the imports to remove Capture and newCapture if no longer used.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In
`@plugin/trino-base-jdbc/src/main/java/io/trino/plugin/jdbc/expression/RewriteCoalesce.java`
around lines 38 - 46, Remove the unnecessary Capture and simplify the pattern in
RewriteCoalesce: delete the static CALL and its newCapture() usage, change the
pattern to just call().with(functionName().matching(name ->
name.equals(COALESCE_FUNCTION_NAME))), and in the rewrite(Call call, ...)
implementation use the provided call parameter instead of using CALL.get(). Also
remove unused imports Capture and newCapture from the file.

57-72: Remove redundant verify and use call parameter directly.

  1. Line 61 uses captures.get(CALL).getArguments() but call.getArguments() is equivalent and simpler.
  2. The verify at line 71 is redundant—if line 57 passes with N arguments, the loop will produce exactly N rewritten arguments (or return early on failure).
♻️ Suggested simplification
     `@Override`
     public Optional<ParameterizedExpression> rewrite(Call call, Captures captures, RewriteContext<ParameterizedExpression> context)
     {
         verify(call.getArguments().size() >= 2, "Function 'coalesce' expects more than or equals to two arguments");
 
         ImmutableList.Builder<String> rewrittenArguments = ImmutableList.builderWithExpectedSize(call.getArguments().size());
         ImmutableList.Builder<QueryParameter> parameters = ImmutableList.builder();
-        for (ConnectorExpression expression : captures.get(CALL).getArguments()) {
+        for (ConnectorExpression expression : call.getArguments()) {
             Optional<ParameterizedExpression> rewritten = context.defaultRewrite(expression);
             if (rewritten.isEmpty()) {
                 return Optional.empty();
             }
             rewrittenArguments.add(rewritten.get().expression());
             parameters.addAll(rewritten.get().parameters());
         }
 
         List<String> arguments = rewrittenArguments.build();
-        verify(arguments.size() >= 2, "Function 'coalesce' expects more than or equals to two arguments");
         return Optional.of(new ParameterizedExpression("COALESCE(%s)".formatted(join(",", arguments)), parameters.build()));
     }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In
`@plugin/trino-base-jdbc/src/main/java/io/trino/plugin/jdbc/expression/RewriteCoalesce.java`
around lines 57 - 72, In RewriteCoalesce, simplify the argument rewriting loop
by iterating over call.getArguments() instead of
captures.get(CALL).getArguments(), set
ImmutableList.builderWithExpectedSize(call.getArguments().size()) for
rewrittenArguments, and remove the redundant verify(arguments.size() >= 2) check
at the end (the initial verify on call.getArguments().size() already guarantees
the count or early-return on rewrite failure); keep the existing behavior of
returning Optional.empty() on rewrite failure and constructing the final
ParameterizedExpression("COALESCE(%s)".formatted(join(",", arguments)),
parameters.build()).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In
`@plugin/trino-base-jdbc/src/main/java/io/trino/plugin/jdbc/expression/RewriteCoalesce.java`:
- Around line 38-46: Remove the unnecessary Capture and simplify the pattern in
RewriteCoalesce: delete the static CALL and its newCapture() usage, change the
pattern to just call().with(functionName().matching(name ->
name.equals(COALESCE_FUNCTION_NAME))), and in the rewrite(Call call, ...)
implementation use the provided call parameter instead of using CALL.get(). Also
remove unused imports Capture and newCapture from the file.
- Around line 57-72: In RewriteCoalesce, simplify the argument rewriting loop by
iterating over call.getArguments() instead of captures.get(CALL).getArguments(),
set ImmutableList.builderWithExpectedSize(call.getArguments().size()) for
rewrittenArguments, and remove the redundant verify(arguments.size() >= 2) check
at the end (the initial verify on call.getArguments().size() already guarantees
the count or early-return on rewrite failure); keep the existing behavior of
returning Optional.empty() on rewrite failure and constructing the final
ParameterizedExpression("COALESCE(%s)".formatted(join(",", arguments)),
parameters.build()).

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: a8b37cf2-38be-4aea-bdc7-a893f1096874

📥 Commits

Reviewing files that changed from the base of the PR and between 535c945 and 72651ec.

📒 Files selected for processing (4)
  • plugin/trino-base-jdbc/src/main/java/io/trino/plugin/jdbc/expression/RewriteCoalesce.java
  • plugin/trino-postgresql/src/main/java/io/trino/plugin/postgresql/PostgreSqlClient.java
  • plugin/trino-postgresql/src/test/java/io/trino/plugin/postgresql/TestPostgreSqlClient.java
  • plugin/trino-postgresql/src/test/java/io/trino/plugin/postgresql/TestPostgreSqlConnectorTest.java

@ebyhr ebyhr force-pushed the ebi/connector-expression-coalesce branch from 72651ec to 0142908 Compare April 6, 2026 21:54
@ebyhr ebyhr merged commit 1c53b9a into trinodb:master Apr 6, 2026
68 checks passed
@ebyhr ebyhr deleted the ebi/connector-expression-coalesce branch April 6, 2026 22:43
@github-actions github-actions bot added this to the 481 milestone Apr 6, 2026
@ebyhr ebyhr mentioned this pull request Apr 7, 2026

List<String> arguments = rewrittenArguments.build();
verify(arguments.size() >= 2, "Function 'coalesce' expects more than or equals to two arguments");
return Optional.of(new ParameterizedExpression("COALESCE(%s)".formatted(join(",", arguments)), parameters.build()));
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this lazy eval for PostgreSQL too?
We don't seem to have any test coverage for this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla-signed postgresql PostgreSQL connector

Development

Successfully merging this pull request may close these issues.

7 participants